1. Field
The present disclosure relates generally to image processing and, more particularly, to stabilization of focal plane data.
2. Related Art
Focal plane arrays are used in a number of image processing applications. For example, missile interceptor guidance techniques may include acquiring and processing digital video data from the focal planes of cameras mounted on the interceptor. The camera optics may provide a way to focus light from distant sources onto the focal planes, which may be two-dimensional staring arrays of many thousands of detector elements. These detector elements may capture light in various wave bands to provide a signal for detecting and tracking targets.
Targets appear as small groups of pixels against a uniform background. At intercept, the target may quickly grow to fill the entire viewable two-dimensional field-of-view (FOV).
It may be desirable to minimize weight, space, and power for focal plane arrays and related components by providing as compact a design as feasible. Simultaneously, it may also be desirable to reduce the unwanted noise so as not to compromise the signal. One such source of noise may be jitter due to unsteadiness of the camera. Other sources of noise may be uncertainties in focal plane characteristics, or read-out and quantization noise.
Reliability of a missile interceptor system may be affected by the signal-to-noise ratio. In particular, increasing the signal-to-noise ratio may be desired to improve the range at which targets can be reliably tracked.
Referring now to FIG. 1, illustrated is an exemplary camera 110 having a target image 120 on a focal plane array of detector elements 130. Inertial measurement unit 140 measures angular data which can be received by the focal plane array 130. At any particular moment, a reference boresight position on the focal plane (i.e. the camera's [0, 0] coordinates), extended as a ray from the camera 110 constitutes the line-of-sight of the camera 110 and may be available from an inertial measurement unit (IMU).
As indicated, stabilization is important in reducing the noise. Some have attempted to address stabilization issues using mechanical gimbal assemblies. Referring now to FIG. 2, illustrated is a camera 200 mounted to a missile interceptor 230 via a gimbal assembly 210.
This gimbal assembly 210 is a mechanical stabilization solution used to stabilize the camera body 200 and focal plane. The gimbal assembly 210 is designed to physically rotate the camera 200 so that the camera remains stable in a reference coordinate system. As an alternative to the gimbal assembly, mirror assemblies may be used to steer light sources into the camera to achieve a similar effect.
Mechanical gimbal assemblies, such as the one shown, may suffer from shortcomings in that they may be heavy and bulky. Moreover, the electronics used in these systems may include closed-loop feedback reaction to the sensed motions of an in-flight missile interceptor. Such electronics may reduce coupling of the interceptor motion and the camera body into a considerably smaller level of jitter in the optical signal.
Nonuniformity in focal plane detectors may also be a source of noise. Therefore, while focal plane detectors may be highly sensitive to incoming light, nonuniformity between detectors may counterbalance detector sensitivity. While calibration may be helpful in reducing the effects of this nonuniformity, a noise component may still exist. This noise component may corrupt the signal as it moves across the detector array.
Some have attempted to address the issue of detector-to-detector nonuniformity using differential techniques or calculations. Referring now to FIG. 3, illustrated is a graphical comparison 300 of a differential technique 310 and a constant technique 320. Results from the differential technique 310 are shown at the upper portion of the screen. Results from the constant technique are shown at the lower portion of the screen. As illustrated, noise may be attributed to the non-uniformity of the detectors.
Differential detection as represented by 310 occurs when an individual detector subtracts previous samples on that detector from current samples. The non-uniformity may then become irrelevant because the measurements are taken against the same detector. The inherent sensitivity of the detector, subject to its read-out noise, may be realized.
The device on which a camera having a focal plane array is mounted may also produce jitter through its motion. For example, in the case of a missile interceptor, jitter may be caused by a missile interceptor's body motion. This jitter may need to be reduced in order to improve camera optics. This jitter may be reduced using a coordinate transformation on incoming data.
Referring now to FIG. 4, illustrated is a graphical representation of a coordinate system on which a coordinate transformation may be performed. As illustrated, the plane of y and z may be the virtual field-of-view (i.e., the original field-of-view having coordinate transformations).
The camera angles relative to a reference frame in pitch, yaw, and roll, as measured by integration of gyro delta-angle measurements in each axis, may be applied in a succession of operations to re-register the camera data to a virtual field-of-view or VFOV. The virtual field-of-view may merely be a set of numbers which only exists in computation.
The real video data may be captured in frames. Frames may be an entire sample set from the focal plane, and they may be continuously collected from the camera. Frames may be first translated in pitch and yaw, and then rotated in roll to compensate for the instantaneous line of sight difference relative to the reference coordinate system. These measurements are available from the missile's navigation system as either quaternions or direction cosines. For a forward-looking camera, these angles may be the same as missile body angles. The equations for a general transformation of this sort are well known.
Virtual output pixels may be arithmetically calculated based on the proportional intensities of the overlapping real pixels which cover it. Referring now to FIG. 5, illustrated is a virtual focal plane 500 illustrating the mapping of real pixels 510 to virtual pixels 520. Computationally, this may be a complicated irregular calculation. Moreover, this computation combines data from multiple, imperfectly compensated real pixels. The resultant virtual pixel will necessarily incorporate more noise. Further, any direct implementation of computations such as those described may be impractically large using conventional computing techniques. Detector arrays can have in the range of one million pixels and the sampling rates required for reproducing missile motion can be in the hundreds of hertz, making this a formidable computing challenge.
There is a need for a stabilization system that reduces mechanical complexity so that the mechanisms used to stabilize the focal plane data are not so heavy and bulky.
There is further a need for a stabilization system that reduces electronic complexity so that the electronics do not become a significant source of noise that counterbalance the detector sensitivity.
There is still further a need for a stabilization system that performs coordinate transformation without the complexities and irregularities present in some prior art techniques.